Polybutene-1 (PB-1), a typical semicrystalline polymer, in its stable form I shows a peculiar temperature dependent strain-whitening behavior when being stretched at temperatures in between room temperature and melting temperature of the crystallites where the extent of strain-whitening weakens with the increasing of stretching temperature reaching a minima value followed by an increase at higher stretching temperatures. Correspondingly, a stronger strain-hardening phenomenon was observed at higher temperatures. The strain-whitening phenomenon in semicrystalline polymers has its origin of cavitation process during stretching. In this work, the effect of crystalline lamellar thickness and stretching temperature on the cavitation process in PB-1 has been investigated by means of combined synchrotron ultrasmall-angle and wide-angle X-ray scattering techniques. Three modes of cavitation during the stretching process can be identified, namely “no cavitation” for the quenched sample with the thinnest lamellae where only shear yielding occurred, “cavitation with reorientation” for the samples stretched at lower temperatures and samples with thicker lamellae, and “cavitation without reorientation” for samples with thinner lamellae stretched at higher temperatures. The mode “cavitation with reorientation” occurs before yield point where the plate-like cavities start to be generated within the lamellar stacks with normal perpendicular to the stretching direction due to the blocky substructure of the crystalline lamellae and reorient gradually to the stretching direction after strain-hardening. The mode of “cavitation without reorientation” appears after yield point where ellipsoidal shaped cavities are generated in those lamellae stacks with normal parallel to the stretching direction followed by an improvement of their orientation at larger strains. X-ray diffraction results reveal a much improved crystalline orientation for samples with thinner lamellae stretched at higher temperatures. The observed behavior of microscopic structural evolution in PB-1 stretched at different temperatures explains above mentioned changes in macroscopic strain-whitening phenomenon with increasing in stretching temperature and stress-strain curves.